Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Add reflect(skip_serializing) which retains reflection but disables automatic serialization #5250

Closed
wants to merge 23 commits into from

Conversation

makspll
Copy link
Contributor

@makspll makspll commented Jul 8, 2022

Objective

Solution

  • Introduce reflect(skip_serializing) on top of reflect(ignore) which disables automatic serialisation to scenes, but does not disable reflection of the field.

Changelog

  • Adds:
    • bevy_reflect::serde::type_data module
    • SerializationData structure for describing which fields are to be/not to be ignored, automatically registers as type_data for struct-based types
    • the skip_serialization flag for #[reflect(...)]
  • Removes:
    • ability to ignore Enum variants in serialization, since that didn't work anyway

Migration Guide

  • Change #[reflect(ignore)] to #[reflect(skip_serializing)] where disabling reflection is not the intended effect.
  • Remove ignore/skip attributes from enum variants as these won't do anything anymore

@alice-i-cecile
Copy link
Member

Bikeshed: I think I prefer do_not_serialize as a name here.

@alice-i-cecile alice-i-cecile added C-Usability A targeted quality-of-life change that makes Bevy easier to use A-Reflection Runtime information about types labels Jul 8, 2022
@makspll makspll changed the title Add reflect(ignore_serialization) which retains reflection but disables automatic serialization Add reflect(do_not_serialize) which retains reflection but disables automatic serialization Jul 8, 2022
@makspll
Copy link
Contributor Author

makspll commented Jul 8, 2022

done!

@MrGVSV
Copy link
Member

MrGVSV commented Jul 8, 2022

More bike shedding 😄: no_serde or skip_serde.

It's just a bit less to write and cleaner. But do_not_serialize isn't bad either haha

Edit: or use skip_serializing like serde does

@makspll
Copy link
Contributor Author

makspll commented Jul 8, 2022

Hmm, I am against serde mentions since this might make people think (more than it does now) this applies serde's skip_serializing to the field which it doesn't.

Right now this only disables automatic Reflect serialization.

Edit: skip_serializing does sound good too, which one should I go for ?

@alice-i-cecile
Copy link
Member

alice-i-cecile commented Jul 8, 2022

skip_serializing for consistency with serde is my new favorite 😄

@makspll makspll changed the title Add reflect(do_not_serialize) which retains reflection but disables automatic serialization Add reflect(skip_serializing) which retains reflection but disables automatic serialization Jul 8, 2022
Copy link
Member

@alice-i-cecile alice-i-cecile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PROMETHIA-27 @MrGVSV can I get a second review here?

@MrGVSV
Copy link
Member

MrGVSV commented Sep 3, 2022

This probably needs to be rebased. @makspll would you be able to do that?

@makspll
Copy link
Contributor Author

makspll commented Sep 4, 2022

I am currently on holiday, but can get on this ~monday

@makspll makspll force-pushed the no-serialize-attribute-reflect branch from 95dc399 to cf9b287 Compare September 8, 2022 10:32
@makspll
Copy link
Contributor Author

makspll commented Sep 8, 2022

Aight, so I am working on this now, it's been a while so I'll take my time with this (and reflection changed a lot by the looks of it) but one thing I noticed so far:
when running the scene example,

bevy_asset::asset_server: encountered an error while loading an asset: No registration found for `glam::vec3::Vec3`

Now I can't remember if that was there before, probably was but I didn't notice, what I think is happening is that because of the way glam imports changed, the type names are now platform dependent, meaning that on my machine the type name this is saved under is now: glam::f32::vec3::Vec3, this is not a problem with this PR in that case and it might have already been addressed somewhere else (and might be fixed after I re-base) but I am just gonna leave this mental note here.

I've squashed all the commits for ease of re-basing, apologies for the force push

@MrGVSV
Copy link
Member

MrGVSV commented Sep 8, 2022

Now I can't remember if that was there before, probably was but I didn't notice, what I think is happening is that because of the way glam imports changed, the type names are now platform dependent, meaning that on my machine the type name this is saved under is now: glam::f32::vec3::Vec3, this is not a problem with this PR in that case and it might have already been addressed somewhere else (and might be fixed after I re-base) but I am just gonna leave this mental note here.

Yep this is a known issue and we're trying to fix it via #5805.

@makspll
Copy link
Contributor Author

makspll commented Sep 8, 2022

Yep this is a known issue and we're trying to fix it via #5805.

Okay that's fantastic, I remember bumping into that problem earlier with scripting but it completely fell out my head

@makspll
Copy link
Contributor Author

makspll commented Sep 8, 2022

One question @MrGVSV, are enum variants supposed to be ignorable by reflection/serialization ?

If so, how would the serialziation/deserialization of ignored variants look like?

@MrGVSV
Copy link
Member

MrGVSV commented Sep 8, 2022

One question @MrGVSV, are enum variants supposed to be ignorable by reflection/serialization ?

I added it as an option. I don't know what specific use-cases it has, though. My original thought was that it would allow for enums to contain variants meant for runtime data and variants meant for serialized data, such as HandleId.

If you find that it's not necessary or should be removed, feel free to make an issue for it, though! I have no problem removing it if it isn't useful haha

@makspll
Copy link
Contributor Author

makspll commented Sep 9, 2022

One question @MrGVSV, are enum variants supposed to be ignorable by reflection/serialization ?

I added it as an option. I don't know what specific use-cases it has, though. My original thought was that it would allow for enums to contain variants meant for runtime data and variants meant for serialized data, such as HandleId.

If you find that it's not necessary or should be removed, feel free to make an issue for it, though! I have no problem removing it if it isn't useful haha

Hmm I see, Yeah I definitely see possible uses for it so removing is not what I want to do. I am wondering however if the current reflect(ignore) works at all (unless I am missing something, and I very well might be, I just didn't see tests for this)? Since the deserialization requires a variant field, which one can only access at serialization IFF the variant is not ignored (the macros don't process inactive variants at all). The solution is probably modifying the macros to generate variant names for all variants regardless if they are ignored.

First bit of serialization:

        let variant_type = self.enum_value.variant_type(); // crash here on ignored variants
        let variant_name = self.enum_value.variant_name();

First bit of deserialization:

        let key = map.next_key::<String>()?;
        match key.as_deref() {
            Some(type_fields::VARIANT) => {}
            Some(key) => return Err(V::Error::unknown_field(key, &[type_fields::VARIANT])),
            _ => {
                return Err(V::Error::missing_field(type_fields::VARIANT));
            }
        }

Relevant macro code on main:

  for variant in reflect_enum.active_variants() {
        let ident = &variant.data.ident;
        let name = ident.to_string();
        let unit = reflect_enum.get_unit(ident);

       // ... more code ...

        let mut add_fields_branch = |variant, info_type, arguments, field_len| {
            // .. more code ..
            // note this code doesn't run for ignored variants
            enum_field_len.push(quote! {
                #unit{..} => #field_len
            });
            enum_variant_name.push(quote! {
                #unit{..} => #name
            });
            enum_variant_type.push(quote! {
                #unit{..} => #bevy_reflect_path::VariantType::#variant
            });
        };

@MrGVSV
Copy link
Member

MrGVSV commented Sep 14, 2022

Hmm I see, Yeah I definitely see possible uses for it so removing is not what I want to do. I am wondering however if the current reflect(ignore) works at all (unless I am missing something, and I very well might be, I just didn't see tests for this)? Since the deserialization requires a variant field, which one can only access at serialization IFF the variant is not ignored (the macros don't process inactive variants at all). The solution is probably modifying the macros to generate variant names for all variants regardless if they are ignored.

Oh great catch! Yeah, it's a bit weird because methods like Enum::variant_type work directly on the type, making it difficult to partially ignore. I see three options:

  1. Do like you suggest, and just don't ignore the variants for any method that relies on current variant info
  2. Have those methods return sensible defaults rather than throwing unreachable!() (e.g. return Option<&str> rather than &str)
  3. Do not allow variants to be ignored

While I see the possible usefulness of ignoring variants, perhaps it's better to just go with Option 3 and remove this functionality altogether (at least for the time being). Option 1 isn't awful but I think it will have a breadth that basically matches Option 3. Option 2 just hurts ergonomics imo.

I think it will be better to remove this as it's a really exposed footgun (oversight on my part). We can always add it back in, and it would be better to remove it now before 0.9 releases with it (forcing a possible future removal to be a breaking change).

I'll make an issue for this unless there's any pushback.

@makspll
Copy link
Contributor Author

makspll commented Sep 14, 2022

@MrGVSV yeah okay, I haven't touched much code in a month so I assumed I was tripping :P

Alright well, In that case I am in favour of removing the variant ignore behaviour for now on the basis of what's already been said + the fact that adding it complicates this PR a bit:

without variant serialization, I can represent "Serialization Ignoring Behaviour" with a set of field indices, with variants I now need an Enum, with either that or a set of strings for variant names. Keeping just index sets is probably more performant and ergonomic, since the serialization process is just:

for (idx,field) in my_thing.iter_fields().enumerate(){
  if !ignored.contains(idx){
     // serialize 
  }
}

@makspll
Copy link
Contributor Author

makspll commented Sep 15, 2022

Alright so that should be it then, I've removed the variant ignoring mechanisms, and re-based everything, should be ready for review

@makspll
Copy link
Contributor Author

makspll commented Sep 18, 2022

Alrighty, that should be @MrGVSV's review implemented, biggest change is SerializationData is now under data in the registry

Copy link
Member

@MrGVSV MrGVSV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few more comments

crates/bevy_reflect/src/serde/mod.rs Outdated Show resolved Hide resolved
crates/bevy_reflect/src/serde/type_data.rs Outdated Show resolved Hide resolved
crates/bevy_reflect/src/serde/type_data.rs Outdated Show resolved Hide resolved
makspll and others added 4 commits September 18, 2022 17:21
Co-authored-by: Gino Valente <49806985+MrGVSV@users.noreply.github.com>
Co-authored-by: Gino Valente <49806985+MrGVSV@users.noreply.github.com>
Copy link
Member

@MrGVSV MrGVSV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@makspll
Copy link
Contributor Author

makspll commented Sep 18, 2022

Fantastic,
This PR will of course need a follow up updating the existing #[reflect(ignore)] and swapping to #[reflect(skip_serializing)] where possible, I am happy to do that after this PR is merged

Copy link
Member

@alice-i-cecile alice-i-cecile left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me now! Those dead_code annotations are surprising to me, but they do seem to be needed.

@alice-i-cecile alice-i-cecile added the S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it label Sep 19, 2022
@alice-i-cecile
Copy link
Member

bors r+

bors bot pushed a commit that referenced this pull request Sep 19, 2022
… automatic serialization (#5250)

# Objective

- To address problems outlined in #5245

## Solution

- Introduce `reflect(skip_serializing)` on top of `reflect(ignore)` which disables automatic serialisation to scenes, but does not disable reflection of the field.

---

## Changelog
- Adds: 
  - `bevy_reflect::serde::type_data` module
  - `ReflectSerializableWithData` trait for retrieving `SerializationData` which is implemented via macros
  - `SerializationData` structure for describing which fields are to be/not to be ignored
  - the `skip_serialization` flag for `#[reflect(...)]`
 - Removes:
   - ability to ignore Enum variants in serialization, since that didn't work anyway   
 

## Migration Guide
- Change `#[reflect(ignore)]` to `#[reflect(skip_serializing)]` where disabling reflection is not the intended effect.
- Remove ignore/skip attributes from enum variants as these won't do anything anymore
@bors
Copy link
Contributor

bors bot commented Sep 19, 2022

Build failed (retrying...):

bors bot pushed a commit that referenced this pull request Sep 19, 2022
… automatic serialization (#5250)

# Objective

- To address problems outlined in #5245

## Solution

- Introduce `reflect(skip_serializing)` on top of `reflect(ignore)` which disables automatic serialisation to scenes, but does not disable reflection of the field.

---

## Changelog
- Adds: 
  - `bevy_reflect::serde::type_data` module
  - `SerializationData` structure for describing which fields are to be/not to be ignored, automatically registers as type_data for struct-based types
  - the `skip_serialization` flag for `#[reflect(...)]`
 - Removes:
   - ability to ignore Enum variants in serialization, since that didn't work anyway   
 

## Migration Guide
- Change `#[reflect(ignore)]` to `#[reflect(skip_serializing)]` where disabling reflection is not the intended effect.
- Remove ignore/skip attributes from enum variants as these won't do anything anymore
@bors bors bot changed the title Add reflect(skip_serializing) which retains reflection but disables automatic serialization [Merged by Bors] - Add reflect(skip_serializing) which retains reflection but disables automatic serialization Sep 19, 2022
@bors bors bot closed this Sep 19, 2022
james7132 pushed a commit to james7132/bevy that referenced this pull request Oct 19, 2022
… automatic serialization (bevyengine#5250)

# Objective

- To address problems outlined in bevyengine#5245

## Solution

- Introduce `reflect(skip_serializing)` on top of `reflect(ignore)` which disables automatic serialisation to scenes, but does not disable reflection of the field.

---

## Changelog
- Adds: 
  - `bevy_reflect::serde::type_data` module
  - `SerializationData` structure for describing which fields are to be/not to be ignored, automatically registers as type_data for struct-based types
  - the `skip_serialization` flag for `#[reflect(...)]`
 - Removes:
   - ability to ignore Enum variants in serialization, since that didn't work anyway   
 

## Migration Guide
- Change `#[reflect(ignore)]` to `#[reflect(skip_serializing)]` where disabling reflection is not the intended effect.
- Remove ignore/skip attributes from enum variants as these won't do anything anymore
james7132 pushed a commit to james7132/bevy that referenced this pull request Oct 28, 2022
… automatic serialization (bevyengine#5250)

# Objective

- To address problems outlined in bevyengine#5245

## Solution

- Introduce `reflect(skip_serializing)` on top of `reflect(ignore)` which disables automatic serialisation to scenes, but does not disable reflection of the field.

---

## Changelog
- Adds: 
  - `bevy_reflect::serde::type_data` module
  - `SerializationData` structure for describing which fields are to be/not to be ignored, automatically registers as type_data for struct-based types
  - the `skip_serialization` flag for `#[reflect(...)]`
 - Removes:
   - ability to ignore Enum variants in serialization, since that didn't work anyway   
 

## Migration Guide
- Change `#[reflect(ignore)]` to `#[reflect(skip_serializing)]` where disabling reflection is not the intended effect.
- Remove ignore/skip attributes from enum variants as these won't do anything anymore
ItsDoot pushed a commit to ItsDoot/bevy that referenced this pull request Feb 1, 2023
… automatic serialization (bevyengine#5250)

# Objective

- To address problems outlined in bevyengine#5245

## Solution

- Introduce `reflect(skip_serializing)` on top of `reflect(ignore)` which disables automatic serialisation to scenes, but does not disable reflection of the field.

---

## Changelog
- Adds: 
  - `bevy_reflect::serde::type_data` module
  - `SerializationData` structure for describing which fields are to be/not to be ignored, automatically registers as type_data for struct-based types
  - the `skip_serialization` flag for `#[reflect(...)]`
 - Removes:
   - ability to ignore Enum variants in serialization, since that didn't work anyway   
 

## Migration Guide
- Change `#[reflect(ignore)]` to `#[reflect(skip_serializing)]` where disabling reflection is not the intended effect.
- Remove ignore/skip attributes from enum variants as these won't do anything anymore
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Reflection Runtime information about types C-Usability A targeted quality-of-life change that makes Bevy easier to use S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants